Suggestions for a site ripper? [closed]

Posted by blackrobot on Stack Overflow See other posts from Stack Overflow or by blackrobot
Published on 2009-02-06T21:38:42Z Indexed on 2010/04/13 2:42 UTC
Read the original article Hit count: 397

Filed under:

html

|

css

|

JavaScript

|

site-ripper

I need software that will rip a site via HTTP. It needs to download the images, HTML, CSS, and JavaScript as well as organize it in a file system.

Does anyone know of software that does this? Or would I be better off just writing it myself?

© Stack Overflow or respective owner

Related posts about html

Install usblib package - Ubuntu

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I need the package libusb for another package I am installing. I tried the following which seemed to install the package, sudo apt-get install libusb-dev but when I try to install the other package I get, configure: error: Package requirements (libusb-1.0 >= 0.9.1) were not met: No package… >>> More
Prevent malicious vulnerability scan increasing load on a server

as seen on Server Fault - Search for 'Server Fault'
Hi all, this week we have been suffering some malicious vulnerability scans to our servers, increasing the load on them, making them nearly unusable. The attack is easy to defend, just blocking the offending ip, but only after discovering it. Is there any form of prevent it? Is it normal that… >>> More
can't install psycopg2 in my env on mac os x lion

as seen on Server Fault - Search for 'Server Fault'
I tried install psycopg2 via pip in my virtual env, but got this error: ld: library not found for -lpq (full log here: http://pastebin.com/XdmGyJ4u ) I tried install postgres 9.1 from .dmg and via port, (gksks)iMac-Alexander:~ lorddaedra$ locate libpq /Developer/SDKs/MacOSX10.7.sdk/usr/include/libpq /Developer/SDKs/MacOSX10… >>> More
Bitnami redmine error SVN

as seen on Server Fault - Search for 'Server Fault'
I'm installing the Bitnami Redmine stack (redmine + subversion). Firstly I install configure and test it locally (Ubuntu 14.04 LTS). And everything is OK. I install Bitnami stack on server (Red Hat 4.4.7-4) and configure SVN. I commit files into SVN and connect project into Redmine with SVN repository… >>> More
Can the .htaccess file slow down a website to a crawl? If so, are there better ways to solve these problems with different rewrite rules and such?

as seen on Pro Webmasters - Search for 'Pro Webmasters'
here is my htaccess file...... RewriteCond %{REQUEST_URI} ^/patients/billing/FAQ_billing\.html$ [OR] RewriteCond %{REQUEST_URI} ^/patients/billing/getintouch\.html$ RewriteRule ^patients/billing/(.*)\.html$ $1.php [L,NC] RewriteCond %{REQUEST_URI} ^/patients/findadoctor/a\.html$ [OR] RewriteCond… >>> More

Related posts about css

HTMLUNIT Facebook javascript warning

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi I am trying to create simple application that would be able to search people on facebook using given e-mail. I am already able to log into an account using the HTMLunit tool and even create a page, which should throw result of my search. But when I try to print the result as XML the file is missing… >>> More
$this->headLink() includes duplicated script

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, Just like I did before, I used the following code for my new project. <?=$this->headLink()->appendStylesheet('/Layouts/admin/css/button.css');?> <?=$this->headLink()->appendStylesheet('/Layouts/admin/css/inputText.css');?> <?=$this->headLink()->appendStylesheet('/Layouts/admin/css/fancyTable… >>> More
How much multiple style sheets slow down to website?

as seen on Stack Overflow - Search for 'Stack Overflow'
Here is 3 css file (one is only for IE) <link rel="stylesheet" href="css/blueprint/screen.css" type="text/css" media="screen, projection"> <link rel="stylesheet" href="css/blueprint/print.css" type="text/css" media="print"> <!--[if lt IE 8]><link rel="stylesheet" href="css/blueprint/ie… >>> More
mod_pagespeed is rewriting but not combining

as seen on Stack Overflow - Search for 'Stack Overflow'
I have the following problem. I installed mod_pagespeed but i am not getting the results i want! It does rewrite my css and changes the to the cache url but its not combining the css files. I have seached the web and stackoverflow for it but did not find a solution. Here are the tags <link… >>> More
CSS import or multiple CSS files

as seen on Stack Overflow - Search for 'Stack Overflow'
I originally wanted to include a .css in my HTML doc that loads multiple other .css files in order to divide up some chunks of code for development purposes. I have created a test page: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional… >>> More